Image capturing control apparatus and control method therefor

ABSTRACT

An image capturing control apparatus including an acquisition unit configured to acquire information about a zoom magnification, a combining unit configured to combine a plurality of images acquired by a series of image capturing operations to generate a combined image wider in angle of view than each acquired image, and a control unit configured to perform control to display, before starting of the series of image capturing operations, an item indicating an angle of view of a combined image which is able to be generated by the combining unit, based on the acquired information.

BACKGROUND Field of the Disclosure

Aspects of the present disclosure generally relate to an image capturingcontrol apparatus which is capable of performing a series of imagecapturing operations to combine a plurality of images and to a controlmethod therefor.

Description of the Related Art

In panoramic image capturing, which performs a plurality of imagecapturing operations and combines captured images, there is a method ofdisplaying a guide. Japanese Patent Application Laid-Open No. 6-303562discusses a method of calculating a movement amount of an image havingmoved from the time when an image capturing operation for panoramicimage capturing starts and a movement amount of an image required forgenerating a panoramic image and then displaying a state of progress ofpanoramic image capturing.

The method discussed in Japanese Patent Application Laid-Open No.6-303562 does not enable recognizing the angle of view of a combinedimage to be obtained by combining images in panoramic image capturing,but only enables knowing, after completion of image capturing, whatangle of view has been acquired in panoramic image capturing. Moreover,the method does not enable knowing, before performing panning, at whatspeed to move the camera.

SUMMARY

Aspects of the present disclosure are generally directed to solving atleast one of the above-mentioned issues and providing an image capturingcontrol apparatus which enables the user to perform panoramic imagecapturing in a more appropriate manner.

According to an aspect of the present disclosure, there is provided animage capturing control apparatus including an acquisition unitconfigured to acquire information about a zoom magnification, acombining unit configured to combine a plurality of images acquired by aseries of image capturing operations to generate a combined image widerin angle of view than each acquired image, and a control unit configuredto perform control to display, before starting of the series of imagecapturing operations, an item indicating an angle of view of a combinedimage which is able to be generated by the combining unit combining aplurality of images, based on the acquired information.

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are appearance diagrams of a digital camera serving asan example of an apparatus to which a configuration of an exemplaryembodiment of the present disclosure can be applied.

FIG. 2 is a block diagram illustrating a configuration example of thedigital camera to which a configuration of the present exemplaryembodiment can be applied.

FIG. 3 is a flowchart of panoramic image capturing processing in thepresent exemplary embodiment,

FIGS. 4A, 4B, 4C, 4D, 4E, 4F1, 4F2, 4F3, 4F4, and 4F5 are diagrams whichare used to explain panoramic image capturing.

FIGS. 5A, 5B, 5C, 5D, 5E, 5F, 5G, 5H, 5I, and 5J are diagramsillustrating display examples in the present exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the presentdisclosure will be described in detail below with reference to thedrawings.

FIGS. 1A and 1B are appearance diagrams of a digital camera serving asan exemplary embodiment of an imaging apparatus to which the presentdisclosure can be applied. FIG. 1A is a front surface perspective viewof the digital camera 100, and FIG. 1B is a back surface perspectiveview of the digital camera 100. In FIGS. 1A and 1B, a display unit 28 isa display unit which displays an image or various pieces of information.A shutter button 61 is an operation unit which is usable to issue animage capturing instruction. A mode selection switch 60 is an operationunit which is usable to switch between various modes. A terminal cover40 is a cover which protects connectors (not illustrated) such asconnection cables which connect connection cables of an external deviceand the digital camera 100. A main electronic dial 71 is a rotationaloperation member, so that the main electronic dial 71 can be rotated toperform, for example, changing of setting values such as the shutterspeed and aperture value. A power switch 72 is an operation member whichis usable to switch between turning-on and turning-off of the powersource of the digital camera 100. A sub-electronic dial 73 is arotational operation member which is usable to perform, for example,movement of a selection frame or image advance. Arrow keys 74 arefour-way keys in which upper, lower, left, and right portions thereofare being able to be pressed. An operation corresponding to a pressedportion of the arrow keys 74 is able to be performed. A SET button 75 isa push button which is mainly used to, for example, determine aselection item. The shutter button 61 is a button used to issue an imagecapturing instruction, and is usable to issue an image capturinginstruction for a still image and an image capturing instruction for amoving image. Moreover, the shutter button 61 can be pressed to startpanoramic image capturing. In panoramic image capturing, when theshutter button 61 is pressed, a plurality of images is captured.Moreover, as described below, a plurality of captured images is combinedto generate a horizontally long or vertically long image. The maximumwidth of an image obtained by combining images is predetermined, sothat, when image capturing for a predetermined amount (pixel width) iscompleted, panoramic image capturing ends. Alternatively, when the userpresses the shutter button 61 after panoramic image capturing isstarted, panoramic image capturing ends and a combined image isgenerated from a plurality of images captured until then. A zoom lever78 is a lever located in such a way as to surround the shutter button61, and can be rotated from side to side to move the position of thelens, thus enabling changing the optical zoom magnification. The shutterbutton 61, the main electronic dial 71, the power switch 72, thesub-electronic dial 73, the arrow keys 74, and the SET button 75 areincluded in an operation unit 70. An eyepiece viewfinder 16 is alooking-into type viewfinder which allows the user to observe a focusingscreen (not illustrated) to check focusing or composition of an opticalimage of a subject obtained through a lens unit 150 (FIG. 2).

FIG. 2 is a block diagram illustrating a configuration example of thedigital camera 100 according to the present exemplary embodiment. InFIG. 2, the lens unit 150 is a lens unit in which an interchangeableimage capturing lens is mounted.

A lens 103 is usually composed of a plurality of lens elements, but, inFIG. 2, is illustrated simply as a single lens element. A communicationterminal 6 is a communication terminal used for the lens unit 150 toperform communication with the digital camera 100, and a communicationterminal 10 is a communication terminal used for the digital camera 100to perform communication with the lens unit 150. The lens unit 150performs communication with a system control unit 50 via thecommunication terminals 6 and 10, and causes a lens system controlcircuit 4, which is included in the lens unit 150, to perform control ofa diaphragm 102 via a diaphragm drive circuit 2 and vary the position ofthe lens 103 via an autofocus (AF) drive circuit 3, thus adjustingfocus.

A shutter 101 is provided to control the exposure time of an imagingunit 22 under the control of the system control unit 50. The imagingunit 22 is an image sensor configured with, for example, acharge-coupled device (CCD) element or a complementary metal-oxidesemiconductor (CMOS) element, which converts an optical image into anelectrical signal.

An analog-to-digital (A/D) converter 23 converts an analog image signaloutput from the imaging unit 22 into digital data, and an imageprocessing unit 24 performs predetermined pixel interpolation, resizingprocessing such as reduction, and color conversion processing on thedata output from the A/D converter 23 or data read out from a memorycontrol unit 15. Moreover, the image processing unit 24 performspredetermined calculation processing using the captured image data, andthe system control unit 50 performs exposure control and distancemeasurement control based on the obtained calculation result. With this,autofocus (AF) processing of the through-the-lens (TTL) type, automaticexposure (AE) processing, and electronic flash (EF) (flash preliminarylight emission) processing are performed. The image processing unit 24further performs predetermined calculation processing using the capturedimage data, and performs automatic white balance (AWB) processing of theTTL type based on the obtained calculation result.

Data output from the A/D converter 23 is then written in a memory 32 viathe image processing unit 24 and the memory control unit 15 or directlyvia the memory control unit 15 without via the image processing unit 24.The memory 32 stores image data acquired by the imaging unit 22 andconverted into digital data by the A/D converter 23 or image data thatis to be displayed on the display unit 28. The memory 32 has a storagecapacity to store a predetermined number of still images or a movingimage and sound taken for a predetermined time. Moreover, the memory 32also serves as a memory for image display (video memory).

A digital-to-analog (D/A) converter 13 converts data for image displaystored in the memory 32 into an analog signal and supplies the analogsignal to the display unit 28. In this way, image data for displaystored in the memory 32 is then displayed on the display unit 28 via theD/A converter 13. The display unit 28 performs display corresponding tothe analog signal supplied from the D/A converter 13 on a displaydevice, such as a liquid crystal display (LCD). A digital signalobtained by A/D conversion performed once by the A/D converter 23 andstored in the memory 32 is then converted into an analog signal by theD/A converter 13, and the analog signal is then sequentially transferredto the display unit 28 and displayed thereon, so that the function of anelectronic viewfinder is implemented, thus enabling performingthrough-image display (live view display).

A non-volatile memory 56 is a memory serving as a recording medium onwhich erasing, recording, and reading operations can be electricallyperformed by the system control unit 50, and includes, for example, anelectrically erasable programmable read-only memory (EEPROM). Forexample, constants and a program for operations of the system controlunit 50 are stored in the non-volatile memory 56. The program as usedhere is a computer program which is executed to perform variousflowcharts described below in the present exemplary embodiment.

The system control unit 50 contains therein at least one processor, andcontrols the entire digital camera 100. The system control unit 50functions as a computer which implements various processing operationsin the present exemplary embodiment by executing the program recorded onthe non-volatile memory 56.

A system memory 52 includes a random access memory (RAM), on which, forexample, constants and variables for operations of the system controlunit 50 and a program read out from the non-volatile memory 56.Moreover, the system control unit 50 also performs display control bycontrolling, for example, the memory 32, the D/A converter 13, and thedisplay unit 28.

The mode selection switch 60 switches an operation mode of the systemcontrol unit 50 to any one of, for example, a still image recordingmode, a moving image capturing mode, and a playback mode. Modes includedin the still image recording mode include an automatic image capturingmode, an automatic scene discrimination mode, a manual mode, a panoramicimage capturing mode, various scene modes in which image capturingsettings for the respective image capturing scenes are performed, aprogram automatic exposure (AE) mode, and a custom mode. The modeselection switch 60 is used to directly switch the operation mode of thesystem control unit 50 to any one of these modes included in a menuscreen. Alternatively, after switching to the menu screen is onceperformed by the mode selection switch 60, another operation member canbe used to perform switching to any one of these modes included in themenu screen. Likewise, a plurality of modes can be included in themoving image capturing mode.

A first shutter switch 62 is configured to be turned on in response to ahalfway operation, in other words, a half-pressed state, of the shutterbutton 61 of the digital camera 100 (an image capturing preparationinstruction), thus generating a first shutter switch signal SW1. Inresponse to the first shutter switch signal SW1, operations, such asautofocus (AF) processing, automatic exposure (AE) processing, automaticwhite balance (AWB) processing, and flash preliminary emission (EF)processing, are started.

A second shutter switch 64 is configured to be turned on in response toa complete operation, in other words, a fully-pressed state, of theshutter button 61 (an image capturing instruction), thus generating asecond shutter switch signal SW2. In response to the second shutterswitch signal SW2, the system control unit 50 starts a series of imagecapturing processing operations starting with a still image capturingoperation performed by the imaging unit 22 and a signal readoutoperation from the imaging unit 22 and leading to a writing operationfor data to a recording medium 200.

Various operation members of the operation unit 70 are assigned therespective functions for each situation as appropriate with variousfunction icons displayed on the display unit 28 being selected andoperated, and thus act as various function buttons. The function buttonsinclude, for example, an end button, a back button, an image forwardingbutton, a jump button, a stop-down button, and an attribute changingbutton. For example, when a menu button is pressed, various settablemenu screens are displayed on the display unit 28. The user is allowedto intuitively perform various settings with use of a menu screendisplayed on the display unit 28 and 4-way up, down, left, and rightbuttons and the SET button.

A power source control unit 80 is configured with, for example, abattery detection circuit, a DC-DC converter, and a switch circuit forswitching blocks to be energized, and detects the presence or absence ofattachment of a battery, the type of a battery, and the remaining amountof battery power. Moreover, the power source control unit 80 controlsthe DC-DC converter based on a result of such detection and aninstruction from the system control unit 50, supplies voltages tovarious portions, including the recording medium 200, for respectiveperiods. The power switch 72 is a switch used to switch power-on andpower-off of the digital camera 100. The power source unit 30 includes,for example, a primary battery, such as an alkaline battery or a lithiumbattery, a secondary battery, such as a nickel-cadmium (NiCd) battery, anickel-metal hydride (NiMH) battery, or a lithium (Li) battery, or analternating current (AC) adapter.

A recording medium interface (I/F) 18 is an interface with the recordingmedium 200, such as a memory card or a hard disk. The recording medium200 is a non-volatile recording medium, such as a memory card, which isused to record an image at the time of image capturing, and isconfigured with, for example, a semiconductor memory, an optical disc,or a magnetic disc.

A communication unit 54 connects to an external device or a network viawireless or via a wired cable, and performs transmission and receptionof, for example, a video signal and an audio signal. The communicationunit 54 is also able to connect to a wireless local area network (LAN)or the Internet. The communication unit 54 is able to transmit an image(including a through-image) captured by the imaging unit 22 and an imagerecorded on the recording medium 200, and is also able to receive imagedata or various other pieces of information from an external device.

An orientation detection unit 55 detects the orientation of the digitalcamera 100 with respect to the direction of gravitational force. Whetheran image captured by the imaging unit 22 is an image captured with thedigital camera 100 held in a landscape orientation or an image capturedwith the digital camera 100 held in a portrait orientation can bediscriminated based on the orientation detected by the orientationdetection unit 55. The system control unit 50 is able to appendorientation information corresponding to the orientation detected by theorientation detection unit 55 to an image file of the image captured bythe imaging unit 22 or to perform recording with an image rotated. Theorientation detection unit 55 to be used includes, for example, anacceleration sensor and a gyroscope sensor.

Next, panoramic image capturing processing in the present exemplaryembodiment is described with reference to FIG. 3. This processing isimplemented by the system control unit 50 loading a program recorded onthe non-volatile memory 56 onto the system memory 52 and executing theprogram. Furthermore, this processing is started in response to thedigital camera 100 being powered on and being switched to a panoramicimage capturing mode.

In step S301, the system control unit 50 displays a live view imageacquired by the imaging unit 22 on the display unit 28. The live viewimage is displayed as illustrated in FIG. 5A. In panoramic imagecapturing, the user moves the digital camera 100 to the right or left orupward or downward (performs panning) to gradually change subjects forimage capturing (image capturing angles), so that the captured imagesare combined to generate a combined image with a wide angle of view. Amethod of acquiring an image in panoramic image capturing is describedwith reference to FIGS. 4A, 4B, 4C, and 4D. FIGS. 4A to 4D are diagramsused to explain a relationship between the direction in which to pan thedigital camera 100 and a cropping area to be cropped from image data.FIG. 4A illustrates an effective image region of an image sensorincluded in the imaging unit 22, in which “Wv” denotes the number ofeffective pixels in the horizontal direction and “Hv” denotes the numberof effective pixels in the vertical direction. FIG. 4B illustrates acropping area cropped from image data of the captured image, in which“Wcrop” denotes the number of cropping pixels in the horizontaldirection and “Hcrop” denotes the number of cropping pixels in thevertical direction. FIG. 4C is a diagram illustrating a cropping areawith respect to image data in a case where image capturing is performedwhile the digital camera 100 is panned in horizontal directionsindicated by arrows. In FIG. 4C, an area S1 indicated by hatchingrepresents a cropping area to be cropped from the image data, in whichcropping of the image data in the horizontal direction is set to“Wv>Wcrop” and cropping of the image data in the vertical direction isset to “Hv=Hcrop”. FIG. 4D is a diagram illustrating a cropping areawith respect to image data in a case where image capturing is performedwhile the digital camera 100 is panned in vertical directions indicatedby arrows. In FIG. 4D, an area S2 indicated by hatching represents acropping area to be cropped from the image data, in which cropping ofthe image data in the horizontal direction is set to “Wv=Wcrop” andcropping of the image data in the vertical direction is set to“Hv>Hcrop”. A cropping area of image data of a captured image can be setdifferent for every piece of image data. Moreover, with respect to imagedata to be obtained at the time of start of panning and image data to beobtained at the time of end of panning, the respective cropping areascan be set wider so as to widen the angle of view. The method ofdetermining a cropping area of image data can be set by, for example, adifference between the angle of the digital camera 100 obtainedimmediately after image capturing and the angle of the digital camera100 obtained one frame before. Cropping and storing image data forcombining processing for a panoramic image enables saving a storagecapacity of the memory 32.

In this way, images to be combined are acquired while panning is beingperformed, and a plurality of images is sequentially combined in thehorizontal direction or in the vertical direction in such a manner thatthe same subject becomes situated at the same position. With this, arange (angle of view) which is wider than a range that is able to beimage-captured once in the image sensor can be acquired. In step S301,an item which indicates moving the digital camera 100 to the right, suchas an item 502 illustrated in FIG. 5A, can be displayed.

In step S302, the system control unit 50 acquires optical systeminformation. The optical system information to be acquired in step S302is focal length information about the lens 103 relative to the imagingunit 22, which is acquired based on information about the lens 103attached to the digital camera 100 and the value of a zoom magnificationset thereto. Specifically, the system control unit 50 acquires imagecapturing information concerning the focal length loaded in the systemmemory 52.

In step S303, the system control unit 50 calculates the angle of view ofa panoramic image. The system control unit 50 calculates, asangle-of-view information, the magnitude of the angle of view of acombined image recordable as one panoramic image, based on the focallength information included in the optical system information acquiredin step S302. A specific calculation method is described with referenceto FIG. 4E. FIG. 4E is a diagram schematically illustrating arelationship between the focal length of a lens relative to an imagesensor and the angle of view. There are illustrated an image sensor 401,an image capturing lens 402, and a focal length 403 of the imagecapturing lens 402 relative to the image sensor 401. Then, there is alsoillustrated an angle of view 404 taken by the image sensor 401 and theimage capturing lens 402. Moreover, in the present example, panoramicimage capturing in the horizontal direction is described. Here, assumingthat, with regard to the size of the image sensor 401, the width isdenoted by Sw (mm), the angle of view 404 is denoted by θ (rad), and thefocal length 403 is denoted by r (mm), the angle of view 404 can hecalculated by the following formula (1).

θ=2×arctan((Sw/2)/r)  (1)

The angle of view 404 indicated here is an angle of view relative to theentire surface of the image sensor 401 and, therefore, indicates themagnitude of the angle of view of one captured image used for combining.

On the other hand, the proportion of the angle of view of the entiresurface of the image sensor 401 to the angle of view available forrecording as a panoramic image is equivalent to the proportion of thewidth of one captured image used for combining to the maximum width of arecordable panoramic image. In other words, assuming that the maximumwidth of a recordable panoramic image (panoramic image maximum width) isdenoted by Pw (pixels) and the width of one captured image (one-capturedimage width) is denoted by Iw (pixels), the magnitude of the angle ofview available for recording as one panoramic image can be calculated asthe angle of view Pθ by the following formula (2).

Pθ=(Pw/Iw)×θ  (2)

The following formula (3) is obtained based on formula (1) and formula(2).

Pθ=2×(Pw/Iw)×arctan((Sw/2)/r)  (3)

In other words, in a case where the panoramic image maximum width Pw,the one-captured image width Iw, and the image sensor width Sw areassumed to be uniquely determined according to the digital camera 100,the magnitude Pθ of the angle of view available for recording as onepanoramic image depends on the focal length r.

The details thereof are further described based on a specific numericalexample. Suppose that the image sensor of the imaging unit 22 of thedigital camera 100 is a CMOS sensor of the APS-C size, the horizontalwidth of the effective pixel portion thereof is 22.7 mm, and the focallength of the lens 103 relative to the imaging unit 22 is set to 18 mmby the zoom lever 78. Then, assuming that one captured image used forcombining is composed of 5,000 pixels and the maximum width of arecordable panoramic image is 20,000 pixels, the magnitude of the angleof view available for recording as one panoramic image is calculated byformula (3) as an angle of view of about 258°. Likewise, in a case wherethe focal length is 36 mm, the magnitude of the angle of view availablefor recording as one panoramic image is calculated as an angle of viewof about 140°.

These calculation processing operations are performed by the systemcontrol unit 50 and the image processing unit 24. Moreover, pieces offixed data, such as the panoramic image maximum width, the one-capturedimage width, and the image sensor width, which are previously recordedon, for example, the system memory 52 or the recording medium 200, areloaded by the memory control unit 15 and are processed by the imageprocessing unit 24. Information about the focal length, which is updatedand recorded on the memory 32 each time the zoom lever 78 is operated,is also loaded by the memory control unit 15 and is processed by theimage processing unit 24 during calculation processing. The calculatedangle-of-view information is then recorded on the memory 32 via thememory control unit 15. While, in the above-mentioned example, panoramicimage capturing in the horizontal direction has been described, thepresent exemplary embodiment can also be applied to panoramic imagecapturing in the vertical direction. In panoramic image capturing in thevertical direction, the panoramic image maximum width, the one-capturedimage width, and the image sensor width are changed to respective piecesof information about the height, and similar calculation processingoperations can be performed to obtain the angle of view indicating themagnitude of the angle of view available for recording as one panoramicimage.

In step S304, the system control unit 50 displays an item indicating therange of angle of view available for image capturing in panoramic imagecapturing on the display unit 28. The item to be displayed in step S304includes, for example, an item 503 or 504 illustrated in FIGS. 5A or 5B.FIGS. 5A and 5B illustrate live view images obtained in a case whereimage capturing has been performed on the same subject, and, forexample, the focal length in a case where the live view image 501illustrated in FIG. 5A has been captured is assumed to be 18 mm and thefocal length in a case where the live view image 505 illustrated in FIG.5B has been captured is assumed to be 36 mm. In the example described instep S303, the magnitude of the angle of view available for recording asone panoramic image is 258° in a case where the focal length is 18 mm,and the magnitude of the angle of view available for recording as onepanoramic image is 140° in a case where the focal length is 36 mm. Theitem 503 illustrated in FIG. 5A indicates that the angle of view of upto 258° is available for recording as one panoramic image, so that theitem 503 enables the user to understand that a wide angle of view of180° or more can be acquired as a combined image. The item 504illustrated in FIG. 5B indicates that the angle of view of up to 140° C.is available for recording as one panoramic image, so that the item 504enables the user to understand that an angle of view of less than 180°can be acquired as a combined image. in this way, displaying an itemindicating the angle of view acquired in step S303 enables the user tounderstand the following. Specifically, in a case where the maximumwidth of a recordable panoramic image is fixed (for example, 20,000pixels in the above-mentioned example), the user can understand, beforeimage capturing, an image having what angle of view is able to beacquired in the current condition (for example, the focal length and theimage sensor) of the digital camera 100. Since the user is able tounderstand, before performing panoramic image capturing, how large anangle of view out of the range of 360° encompassing the user can beattained, the user is enabled to perform image capturing while imaginingan image to be obtained by combining. For example, even if the userwould like to perform panoramic image capturing including the middle ofa long bridge, in a case where image capturing is performed with use ofa telephoto lens, when image capturing is started with one end of thebridge, the middle may become unable to be contained in a combinedimage. However, since preliminarily indicating the angle of view of animage to be obtained by combining enables understanding an angle of viewthat is able to be acquired, it becomes easy to understand from wherestarting panoramic image capturing enables an intended subject to becontained in a combined image. Alternatively, as illustrated in FIG. 5B,in a case where image capturing is performed at telephoto setting, theitem 504 enables understanding that the maximum angle of view able to beobtained (the range of field of view available for recording as onepanoramic image) is less than 180 degrees. Accordingly, in a case wherethe user intends to cause the range from one end of the bridge to themiddle thereof to be contained in a combined image, the user can changethe settings to the wide-angle side in advance, thus enabling obtaininga combined image with a wider angle of view. Furthermore, the angle ofview of a captured image range that is currently being displayed as alive view image can be displayed together with the item 503 or 504.

In step S305, the system control unit 50 determines whether an imagecapturing starting instruction has been issued. The image capturingstarting instruction can be issued in response to pressing of theshutter button 61. If it is determined that the image capturing startinginstruction has been issued (YES in step S305), the processing proceedsto step S308, and, if not (NO in step S305), the processing proceeds tostep S306.

In step S306, the system control unit 50 determines whether the lensunit 150 has been replaced and the lens 103 mounted therein has beenchanged. If it is determined that the lens 103 has been changed (YES instep S306), the processing returns to step S302, and, if not (NO in stepS306), the processing proceeds to step S307.

In step S307, the system control unit 50 determines whether the opticalzoom magnification has been changed (whether the lens position has beenchanged). The optical zoom magnification is changed according to anoperation performed on the zoom lever 78. If it is determined that theoptical zoom magnification has been changed (YES in step S307), theprocessing returns to step S302, and, if not (NO in step S307), theprocessing returns to step S305. In a case where the optical zoommagnification or the lens 103 has been changed, since the range of angleof view to be calculated in step S303 changes, the processing returns tostep S302, so that the system control unit 50 re-performs processing forcalculating the angle of view.

In step S308, the system control unit 50 performs image capturingprocessing. In the image capturing processing, the imaging unit 22controls the shutter and the diaphragm and then acquires a still image.

In step S309, the system control unit 50 performs panoramic imagecombining processing. Processing in step S308 and step S309 is describedwith reference to FIGS. 4F1, 4F2, 4F3, 4F4, and 4F5. FIG. 4F1 to FIG.4F5 are diagrams used to explain the flow of combining processing for apanoramic image. In FIG. 4F1, dot-hatched areas are areas schematicallyrepresenting a line of trees included in the field of view, and adiagonal-hatched area represents a cropping area in image data. FIG. 4F1illustrates the position of an image first captured at the time ofpressing of the shutter button 61, in which a range 405 indicates animage capturing range acquired by image capturing performed one time(image capturing processing in step S308), so that the field of view isset to one end of a panoramic image intended to be obtained bycombining. FIG. 4F2 and FIG. 4F3 schematically illustrate a state inwhich image capturing is sequentially performed while the digital camera100 is being panned toward the other end of the panoramic image intendedto be obtained by combining. In panoramic image capturing, imagecapturing is sequentially performed on an image by image basis asillustrated in FIG. 4F2 and FIG. 4F3, and combining processing isperformed in step S309 each time and processing in step S310 to stepS312 described below is also performed. Moreover, processing in stepS308 to step S312 is performed until image capturing is ended in stepS312 when it is determined that the width (number of pixels) of an imagewhich is obtained by combining the acquired pieces of image data hasreached the maximum (for example, 20,000 pixels). Here, the user doesnot issue an image capturing instruction in each image capturingoperation, but an image capturing starting instruction to be issued inresponse to pressing of the shutter button 61 needs to be issued at thefirst operation performed once in the image capturing illustrated inFIG. 4F1. FIG. 4F4 illustrates a case where sequential image capturinghas ended when panning of the digital camera 100 has been performed andthe width (number of pixels) of an image which is obtained by combiningthe acquired pieces of image data has reached the maximum (for example,20,000 pixels). In other words, FIG. 4F4 illustrates a state in whichsequential image capturing has ended when it is determined in step S314that image data corresponding to the maximum angle of view calculated instep S303 has been acquired. In this way, image capturing issequentially performed with the image capturing range gradually changeduntil the number of pixels of the maximum width of a recordablepanoramic image is reached while the user is panning the digital camera100. FIG. 4F5 illustrates a panoramic image finally obtained as a resultof sequentially combining a plurality of captured images (still images).The number of pixels of the maximum width of a recordable panoramicimage is determined by the type of a digital camera or by manualselection settings.

Furthermore, in step S310 described above, the system control unit 50calculates the angle of view of a combined image obtained bysequentially combining captured images. A specific calculation method isdescribed. The proportion of the angle of view of one captured image tothe magnitude of the maximum angle of view recorded in the combinedimage data is equivalent to the proportion of the number of pixels ofthe width of one captured image to the number of pixels of the width ofthe combined image. Thus, the angle of view Cθ (rad) indicating themagnitude of the angle of view recorded in the combined image data iscalculated by the following formula (4) based on formula (3) when thenumber of pixels of the width of the combined image is denoted by Cw(pixels).

Cθ=2×(Cw/Iw)×arctan((Sw/2)/r)  (4)

As with the numerical example described with regard to step S303,panoramic image capturing in the horizontal direction is described.Suppose that the maximum width of an image which is able to be recordedin panoramic image capturing for one image is 5,000 pixels, the imagingunit 22 is a CMOS sensor having an effective pixel portion with ahorizontal width of 22.7 mm, and the focal length of the lens 103 is setto 18 mm. If the horizontal width of the combined image data obtained ata time is 2,000 pixels, the range of the field of view recorded in thecombined image is obtained as an angle of view of about 52°. Thesecalculation processing operations are performed by the system controlunit 50 and the image processing unit 24, and pieces of fixed data, suchas the one-captured image width and the image sensor width, which arepreviously recorded on the system memory 52 or the recording medium 200,are loaded by the memory control unit 15 and are processed by the imageprocessing unit 24. With regard to data about the combined image width,the combined image data stored in the memory 32 in step S309 is loadedand acquired by the memory control unit 15 and is processed by the imageprocessing unit 24. With regard to information about the focal length,the information recorded on the memory 32 is also loaded by the memorycontrol unit 15 and is processed by the image processing unit 24. Thecalculated angle-of-view information about the combined image data isthen recorded on the memory 32 via the memory control unit 15.Alternatively, the angle of view can be calculated by multiplying theangle of view calculated in step S303 by a value obtained by dividingthe combined image width (number of pixels) by the maximum image widthof an image able to be obtained by combining.

In step S311, the system control unit 50 updates a progress itemdisplayed to indicate a degree of panoramic progress. The system controlunit 50 updates the progress item based on the magnitude of the maximumangle of view recorded in the combined image data (image data obtainedby combining captured images) calculated in step S310. The progressitems 506 and 507 illustrated in FIGS. 5C and 5D respectively correspondto the items 503 and 504 illustrated in FIGS. 5A and 5B, and the entirewidth of each of the items 503 and 504 indicates the magnitude of theangle of view (range of the field of view) available for recording asone panoramic image. Each of an area 506 a of the progress item 506 andan area 507 a of the progress item 507 indicates the magnitude of theangle of view obtained by combining. Each of the areas 506 a and 507 ais an arc generated as image data to indicate the angle corresponding tothe angle of view acquired in step S310. Thus, as combining of imagesprogresses, the displaying manner of the progress item is changing asmuch as the angle of an arc corresponding to an angle of view newlyadded by combining. In the examples illustrated in FIGS. 5C and 5D, eachof which indicates a case where panning is performed to the right, inorder from the left end toward the right end, the displaying manner ofthe progress item is changing from gray to black as much as the angle ofan arc corresponding to an angle of view newly added by combining. Inother words, the displaying manner of the progress item is graduallychanging in order from one end toward the panning direction of thecamera (the other end). A hatched-line portion 506 b illustrated in FIG.5C is not actually displayed but is illustrated for purposes ofexplanation, and, when a new captured image is added for combining inthe combining processing performed next in step S309, the displayingmanner of an area defined by the angle α of an arc corresponding to theassociated angle of view changes from gray to black. Thus, as the numberof captured images increases and the angle of view obtained by combiningincreases accordingly, the portion the displaying manner of which ischanging becomes larger. Furthermore, in a case where panning isperformed to the left, the displaying manner of the progress item isgradually changing in order from the right end toward the left end.Furthermore, in step S310, instead of calculating the angle of view, theproportion of the image width (number of pixels) of the current combinedimage to the maximum image width of a combined image able to be obtainedby combining can be obtained and the displaying manner of the progressitem can be changed according to the obtained proportion.

In step S312, the system control unit 50 determines whether imagecapturing has ended, Panoramic image capturing ends when the combinedimage has reached the number of pixels available for recording as onepanoramic image and in response to an image capturing ending instructionbeing issued by the user. The combined image data recorded on the memory32 is acquired via the memory control unit 15 and is then compared withthe previously-determined maximum size of a recordable panoramic imageby the image processing unit 24, and, when the combined image datareaches the maximum size, image capturing ends. If it is determined thatimage capturing has ended (YES in step S312), the processing proceeds tostep S313, and, if not (NO in step S312), the processing returns to stepS308. If, in step S312, it is not determined that image capturing hasended, processing in step S308 to step S311, i.e., a series ofprocessing operations of performing image capturing and displaying thedegree of progress, is repeated.

In step S313, the system control unit 50 performs recording processing.More specifically, the system control unit 50 performs processing forrecording the image obtained by combining in step S309 on the recordingmedium 200.

According to the above-described exemplary embodiment, the user isenabled to understand, before panoramic image capturing, the magnitudeof the angle of view available for recording as one panoramic image andthe degree of progress of panoramic image combining. With this, the useris enabled to adjust the starting position of panoramic image capturingor the appropriate focal length prior to image capturing, thus easilyperforming image capturing within an intended range and saving thetrouble of performing unwanted preliminary image capturing.

[Displaying of Panning Speed]

Moreover, the speed during panning to change the direction of theimaging apparatus can be configured to be displayed. When, as inpanoramic image capturing, the user performs panning while holding thedigital camera 100 by hand, even if the width of swinging is the same,as the focal length becomes longer and the angle of view become largeraccording to, for example, an increase of the zoom magnification, theamount of movement of the field of view formed on the image sensorbecomes larger. In other words, since, during image combining, an areaof overlapping with a preceding captured image is used as an overlapwidth, it is to performs subsequent image capturing in such a way as toleave the area of overlapping, so that, as the focal length becomeslonger, the speed of panning is to be decreased. However, it isdifficult for the user to understand what speed is appropriate toperforming panning. Therefore, performing panning at too high speed maycause a failure in panoramic image capturing, and performing panning attoo low speed may cause an excessive performance of image capturing.

Therefore, an item used to expressly indicate an appropriate panningspeed to the user can be displayed together with the range of the angleof view available for recording as one panoramic image. The appropriatepanning speed is obtained based on the angle-of-view informationcalculated in step S303, the maximum size of a panoramic image, the sizeof an image able to be captured by image capturing performed once, andpixel size information about an overlap width for combining. How manyimages are to be captured to generate one panoramic image is calculatedbased on the above-mentioned pieces of information, and an angle atwhich to perform panning for every image capturing is calculated basedon the maximum width of a panoramic combined image. Then, an imagecapturing interval in panoramic image capturing previously defined inthe digital camera 100 is calculated based on the image capturing speedand the combining processing speed. Finally, the panning speed iscalculated based on the image capturing interval and an angle at whichto perform panning between image capturing and image capturing. Then, aspeed guide such as that illustrated in FIGS. 5E or 5F is displayedbased on the calculated panning speed.

FIG. 5E, which corresponds to FIG. 5A, illustrates displaying of a speedguide 508, and FIG. 5F, which corresponds to FIG. 5B, illustratesdisplaying of a speed guide 509. For example, when the focal length inthe case of FIG. 5E is assumed to be 18 mm and the focal length in thecase of FIG. 5F is assumed to be 36 mm, since faster panning is allowedto be performed in the case of the shorter focal length, the arrow ofthe speed guide 508 is longer than that of the speed guide 509, so thatthe speed guide 508 indicates that faster panning is allowed to beperformed. On the other hand, in a case where the speed guide 509 isdisplayed, the user can understand that panning should be performedslowly. Then, such guides are not limited to an arrow-like objectillustrated by an example, but can be an icon which performs ananimation motion at the calculated speed and can be have any form aslong as the calculated appropriate speed can be indicated. For example,the speed can be expressed by a number. Moreover, an indicationindicating the panning speed can be displayed together with an itemindicating the magnitude of the angle of view available for recording asone panoramic image, such as that illustrated in one of FIGS. 5A to 5D.Moreover, the indication indicating the appropriate panning speed can bedisplayed together with an item indicating the current panning speed,thus enabling understanding whether the current panning speed is higheror lower than the appropriate panning speed.

According to the above-described example, the user is enabled tounderstand in advance at what panning speed the user should performpanoramic image capturing. This makes it easier for the user toimplement panning at an appropriate speed, so that the failure ofpanoramic image capturing or the excessive image capturing can beprevented or reduced.

Furthermore, while, in the description of the above-described exemplaryembodiment, indicating the magnitude of the angle of view available forrecording as one panoramic image has been described with use of FIGS. 5Ato 5F, the displaying manner of an item is not limited to theabove-described ones.

FIGS. 5G and 5H illustrate examples of displaying of rectangular bars.An item 510 illustrated in FIG. 5G corresponds to the item 503, and anitem 511 illustrated in FIG. 5H corresponds to the item 504. These itemsare generated as rectangular image data with a length proportional tothe magnitude of the angle of view calculated in step S303, based ontemplate image data about rectangular bars.

FIG. 5I illustrates an example of displaying of a circularthree-dimensional item. A portion corresponding to an angle obtained bysubtracting a fan-like diagonal-hatched portion 513 from an area 512compassing a circle with a width l, in other words, a portioncorresponding to the angle β, indicates the magnitude of the angle ofview available for recording as one panoramic image. After panoramicimage capturing is started, the displaying manner is changing from theoriginal displaying manner as in a dot-hatched portion 514, thusindicating the degree of progress of combining.

FIG. 5J illustrates an example of displaying in which not only themagnitude of the angle of view available for recording as one panoramicimage is indicated but also the degree of progress of combining isindicated by gradually displaying a combined image in a band-likemanner. In FIG. 5J, the magnitude of the angle of view available forrecording as one panoramic image is indicated by an item 503 and thecombined image is displayed in an area 515. During panoramic imagecapturing, the user pans the digital camera 100 in such a way as to movea frame 516 to the right in the area 515. The user is enabled to performpanning while confirming that images are being combined in the area 515.

These icons for guiding are not limited to the arc-like bars orrectangular bars illustrated as examples, but have any form as long asthe calculated angle of view is able to be indicated. Moreover, theseguides only need to be guides able to display the angle of view and,therefore, does not need to be icons such as those described above butcan be items which directly indicate, for example, a numerical valueequivalent to the calculated angle of view.

Furthermore, while, in the above description, an example in which focallength information about the lens is acquired as zoom magnificationinformation has been described, the present disclosure is not limited tothis, but can be applied to the following case. Specifically, forexample, zoom magnification information about electronic zoom can beacquired, and, for example, the panoramic image capturing angle orpanning speed indication to be displayed in a screen can be naturallychanged according to the acquired zoom magnification information.

A variety of control operations performed by the system control unit 50in the above-described exemplary embodiment can be performed by onepiece of hardware. Alternatively, a plurality of pieces of hardware canshare the processing to control the entire apparatus.

While the exemplary embodiments of the present disclosure have beendescribed with use of specific examples, the present disclosure is notlimited to the specific exemplary embodiments and examples, and it is tobe understood that changes and variations can be made without departingfrom the spirit or scope of the present disclosure. The exemplaryembodiments described above are merely examples and can be appropriatelycombined.

Although the case is exemplified in the above embodiments in which thepresent disclosure is applied to the digital camera 100, the presentdisclosure is not limited to this and is applicable to a recordingcontrol apparatus capable of recording the recording data before therecording instruction is issued. Specifically, the present disclosure isapplicable to, for example, a mobile phone terminal, a mobile imageviewer, a printer apparatus provided with a viewfinder, a digital photoframe, a music player, a game machine, or an electronic book reader.

Other Embodiments

Some exemplary embodiments of the present disclosure are implemented byperforming the foregoing processes. Specifically, software implementingthe functions of the above exemplary embodiments can be supplied to asystem or an apparatus via a network or various recording media, and oneor more computers (or a central processing unit (CPU) or a microprocessing unit (MPU)) in the system or the apparatus can read outprogram codes for execution. In this case, the programs and anon-volatile recording medium having the programs stored thereonconstitute sonic exemplary embodiments.

Some exemplary embodiments can also be implemented by a computer of asystem or apparatus that reads out and executes computer executableinstructions (e.g., one or more programs) recorded on a storage medium(which may also be referred to more fully as a ‘non-transitorycomputer-readable storage medium’) to perform the functions of one ormore of the above-described embodiment(s) and/or that includes one ormore circuits (e.g., application specific integrated circuit (ASIC)) forperforming the functions of one or more of the above-describedembodiment(s), and by a method performed by the computer of the systemor apparatus by, for example, reading out and executing the computerexecutable instructions from the storage medium to perform the functionsof one or more of the above-described embodiment(s) and/or controllingthe one or more circuits to perform the functions of one or more of theabove-described embodiment(s). The computer may comprise one or moreprocessors (e.g., central processing unit (CPU), micro processing unit(MPU)) and may include a network of separate computers or separateprocessors to read out and execute the computer executable instructions.The computer executable instructions may be provided to the computer,for example, from a network or the storage medium. The storage mediummay include, for example, one or more of a hard disk, a random accessmemory (RAM), a read-only memory (ROM), a storage of distributedcomputing systems, an optical disk (such as a compact disc (CD), digitalversatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, amemory card, and the like.

While the present disclosure has been described with reference toexemplary embodiments, it is to be understood that the presentdisclosure is not limited to the disclosed exemplary embodiments. Thescope of the following claims is to be accorded the broadestinterpretation so as to encompass all such modifications and equivalentstructures and functions.

This application claims the benefit of Japanese Patent Application No.2017-221289 filed Nov. 16, 2017, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image capturing control apparatus comprising:an acquisition unit configured to acquire information about a zoommagnification; a combining unit configured to combine a plurality ofimages acquired by a series of image capturing operations to generate acombined image wider in angle of view than each acquired image; and acontrol unit configured to perform control to display, before startingof the series of image capturing operations, an item indicating an angleof view of a combined image which is able to be generated by thecombining unit combining a plurality of images, based on the acquiredinformation.
 2. The image capturing control apparatus according to claim1, wherein the control unit performs control to display an itemindicating a first angle of view in a case where a first lens isconnected and to display an item indicating a second angle of viewlarger than the first angle of view in a case where a second lens widerin angle than the first lens is connected.
 3. The image capturingcontrol apparatus according to claim 1, wherein the control unitperforms control to display an item indicating a third angle of view ina case where the zoom magnification is a first magnification and todisplay an item indicating an angle of view larger than the third angleof view in a case where the zoom magnification is a second magnificationwider in angle than the first magnification.
 4. The image capturingcontrol apparatus according to claim 1, further comprising aninstruction unit configured to issue an instruction to start the seriesof image capturing operations, wherein, in response to the series ofimage capturing operations being started, the control unit performscontrol to perform display indicating an angle of view of a combinedimage obtained by combining captured images.
 5. The image capturingcontrol apparatus according to claim 1, wherein the angle of view of thecombined image is displayed together with the item.
 6. The imagecapturing control apparatus according to claim 1, wherein the item is anindication indicating an angle, and differs in magnitude according tothe acquired information.
 7. The image capturing control apparatusaccording to claim 1, wherein the item is a bar.
 8. The image capturingcontrol apparatus according to claim 7, wherein a displaying manner ofthe bar changes from one end thereof toward another end thereofaccording to a number of captured images increasing after the series ofimage capturing operations is started.
 9. The image capturing controlapparatus according to claim 1, wherein a maximum number of pixels of acombined image able to be generated by combining performed by thecombining unit is equal irrespective of the zoom magnification.
 10. Theimage capturing control apparatus according to claim 1, wherein, in theseries of image capturing operations, image capturing is performed whilea direction of an imaging unit is being changed.
 11. The image capturingcontrol apparatus according to claim 1, wherein the information about azoom magnification is a focal length.
 12. The image capturing controlapparatus according to claim 1, wherein the information about a zoommagnification is electronic zoom.
 13. An image capturing controlapparatus comprising: an acquisition unit configured to acquireinformation about a zoom magnification; a combining unit configured tocombine a plurality of images acquired by a series of image capturingoperations in which image capturing is performed while a direction of animaging unit is being changed; and a control a unit configured toperform control to display, before starting of the series of imagecapturing operations, a guide indicating a speed at which to change thedirection of the imaging unit in the series of image capturingoperations, based on the acquired information.
 14. The image capturingcontrol apparatus according to claim 13, wherein the combining unit is acombining unit configured to generate, from the plurality of imagesacquired by the series of image capturing operations, a combined imagewider in angle of view than each acquired image, and wherein the controlunit performs control to display, before starting of the series of imagecapturing operations, an item indicating an angle of view of a combinedimage which is able to be generated by the combining unit, based on theacquired information, together with the guide.
 15. The image capturingcontrol apparatus according to claim 13, wherein the information about azoom magnification is a focal length.
 16. The image capturing controlapparatus according to claim 13, wherein the information about a zoommagnification is electronic zoom.
 17. A non-transitory computer-readablerecording medium having stored thereon a program that causes a computerto function as each unit of the image capturing control apparatusaccording to claim
 1. 18. A non-transitory computer-readable recordingmedium having stored thereon a program that causes a computer tofunction as each unit of the image capturing control apparatus accordingto claim 13.